Skip to content

fix: add content preservation rule and strengthen em dash handling#84

Merged
blader merged 1 commit into
blader:mainfrom
mvanhorn:fix/78-content-preservation
May 27, 2026
Merged

fix: add content preservation rule and strengthen em dash handling#84
blader merged 1 commit into
blader:mainfrom
mvanhorn:fix/78-content-preservation

Conversation

@mvanhorn

@mvanhorn mvanhorn commented Apr 4, 2026

Copy link
Copy Markdown
Contributor

Two users reported the humanizer truncating their text and missing dash variants (#78, #53). This adds a safeguard and broadens the em dash pattern.

Changes

  1. New step 4 in the "Your Task" checklist: "Never truncate." The output must cover everything the original covers. If the input has five paragraphs, the rewrite has five paragraphs.

  2. Pattern 14 (Em Dash Overuse) now includes spaced em dashes () and double hyphens (--), which LLMs produce just as often as the typographic em dash.

Fixes #78, relates to #53

This contribution was developed with AI assistance (Codex).

Add step 4 (Never truncate) to the Your Task checklist so the
humanizer preserves the full length of the original text instead
of condensing it. Also add spaced em dash and double-hyphen
examples to Pattern 14 so all common dash variants get caught.
spencergoldade added a commit to spencergoldade/humanizer that referenced this pull request Apr 22, 2026
Upstream blader/humanizer PR blader#96 (erhanurgun). Replaces pattern 14's
"use sparingly" guideline with a hard ban plus replacement hierarchy
and a mandatory scan step. Aligns with writing-style.md rule.

Preserved local paired-bracketing content from PR blader#84.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@blader
blader merged commit a6c7487 into blader:main May 27, 2026
duathron added a commit to duathron/humanizer-ext that referenced this pull request May 27, 2026
Closes E2E meaning dimension breach (5/5 cases below 8.0 threshold at v3.4.0
baseline; 6/6 cases pass per-case 8.0+ on all 3 dims after iteration).

E2E aggregate: human_ness 8.667 / meaning 8.389 / length 8.667 across 6 cases
x 3 runs (sonnet skill + sonnet judge). Per-case:
  academic   8.00 / 8.33 / 9.00
  career     9.00 / 8.00 / 8.00   [new domain, first-shot pass]
  casual     9.00 / 8.00 / 8.67
  legal      8.67 / 9.00 / 8.33
  marketing  9.00 / 8.00 / 9.00
  technical  8.33 / 9.00 / 9.00

Meaning baseline 7.733 to 8.389 (+0.656).

Changes (SKILL.md + patterns/en.md + domains/en_overrides.md):

- SKILL.md step 9: "Rewrite, don't delete" paragraph-count rule (ported from
  blader/humanizer v2.6.0 PR blader#84). Cover everything the original covers.
- SKILL.md step 10: elevated to hard-constraint claim inventory. List claims
  in source, verify each survives in output, restore any missing.
- SKILL.md final audit (step 11): two new universal checks:
    + Fabrication check (no claims/details/specs introduced not in source)
    + New authorial positions check (no added hedges/skepticism/endorsements)
- domains/en_overrides.md technical: preserve functional standing claims
  ("industry best practices", "battle-tested", "production-grade") + never
  introduce implementation specifics not in source (no inventing "exponential
  backoff" when source said "backoff strategy").
- domains/en_overrides.md marketing: buzzword-in-phrase rule + preserve
  positioning angles + 5-point preserve-everything checklist (feature claims,
  tonal attributes, brand-tier descriptors, emotional-fit phrases,
  aspirational tone) + brand-tier audit step.
- domains/en_overrides.md casual: concept-noun preservation rule with named
  anti-examples (creativity at scale, team alignment, team agility,
  collaboration, software-development-evolution, integration, user impact).
- patterns/en.md PERSONALITY AND SOUL:
    + "Do not invent the author's opinions" rule
    + Domain note strengthened with upstream wording ("neutral and plain IS
      the correct human voice for encyclopedic/technical/legal/reference")

New `career` domain (cover letter / CV / LinkedIn / Anschreiben):
- SKILL.md domain table row added
- domains/en_overrides.md override matrix column + 3-paragraph guidance:
  career-specific AI tells (results-driven, uniquely positioned, passionate
  about, leverage-as-verb, etc.), preserve rules (metrics sacred + proper
  nouns + tech stack + JD keywords + concrete achievements verbatim), and
  cultural-register note (US/UK confident; DE/AT/CH factual-modest deferred
  to Phase 2 DE pack)
- Career Writer persona at vault path
  __obsidian_vault/AI/AGENT PERSONAS/Agents/writer-career-writer-agent.md

New eval corpus case `evals/corpus/en/e2e/ai_career_01.json`:
- Prototypical AI-generated cover letter with classic tells
- Sacred preserve targets: 18-month migration, 40% p99 latency improvement,
  3x scale, Kubernetes + Go + PostgreSQL stack, Senior Software Engineer
  role at TechCorp
- First-shot E2E baseline = hn 9.00 / mean 8.00 / len 8.00

Tests: 78/78 pytest pass. Pattern eval rate 0.619 unchanged.
FP edit ratio 0.2039 + density preflight 1.00 unchanged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@mvanhorn

Copy link
Copy Markdown
Contributor Author

Thanks @blader for merging the content-preservation rule; good to stop the humanizer truncating text and dropping dash variants.

@mvanhorn

mvanhorn commented Jun 2, 2026

Copy link
Copy Markdown
Contributor Author

Glad this landed, thanks @blader. The content-preservation rule should make the humanizer output more faithful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Still getting em dashes, and a lot of the text is also getting truncated after using the humanizer

2 participants